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PREFACE 


This  report  was  prepared  as  part  of  Rand's  DoD  Training  and  Man- 
power Management  Program,  sponsored  by  the  Cybernetics  Technology 
Office  of  the  Defense  Advanced  Research  Projects  Agency  (ARPA) . With 
manpower  issues  assuming  an  ever  greater  importance  in  defense  plan- 
ning and  budgeting,  the  purpose  of  this  research  program  is  to  develop 
broad  strategies  and  specific  solutions  for  dealing  with  present  and 
future  military  manpower  problems.  This  includes  the  development  of 
new  research  methodologies  for  examining  broad  classes  of  manpower 
problems,  as  well  as  specific  problem-oriented  research.  In  addition 
to  providing  analysis  of  current  and  future  manpower  issues,  it  is 
hoped  that  this  research  program  will  contribute  to  a better  general 
understanding  of  the  manpower  problems  confronting  the  Department  of 
Def ense. 

Military  training  has  received  substantial  attention  in  recent 
years  as  one  area  in  which  improved  efficiency  can  lead  to  substan- 
tial cost  reductions.  This  report  treats  an  aspect  of  training  effi- 
ciency that  has  to  date  received  very  little  attention — the  question 
of  the  efficient  amount  of  formal  training.  Specifically,  it  deals 
with  the  efficient  mix  between  formal  and  on-the-job  training  in  pro- 
viding first-term  enlisted  personnel  with  the  skills  needed  to  perform 
effectively  in  their  military  specialties.  Since  formal  initial  spe- 
cialty training  for  first-term  enlisted  personnel  currently  costs 
about  $2  billion  per  year,  increased  efficiency  in  this  area  could 
lead  to  substantial,  recurring  savings.  This  report  describes  problems 
involved  in  analyzing  tradeoffs  between  formal  and  on-the-job  training, 
alternative  solutions  to  these  problems,  and  some  early  results  of  Rand 
research  in  this  area. 

The  substance  of  this  work  was  originally  presented  at  the  Rand 
Conference  on  Defense  Manpower,  February  4-6,  1976,  and  is  Included 
in  the  conference  proceedings:  Defense  Manpower  Policy:  Presentations 
from  the  1976  Rand  Conference  on  Defense  Manpower , edited  by  Richard 
V.  L.  Cooper,  R-2396-ARPA,  December  1978.  The  present  report  is  being 


published  separately  because,  as  one  of  several  planned  publications 
dealing  with  Rand’s  training  and  productivity  research,  it  is. expected 
to  have  an  audience  different  from  that  for  the  conference  proceedings. 
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SUMMARY 


The  U.S.  military  is,  among  other  things,  a very  large  training 
institution.  It  is  estimated  that  the  cost  of  formal  military  train- 
ing during  FY  1976  was  about  $7  billion  and  involved  about  250,000 

* 

man-years  in  student  time.  A substantial  portion  of  this  cost  is 
associated  with  the  initial  specialty  training  given  to  entrants  to 
the  enlisted  force.  Estimates  are  that  in  FY  1976,  over  $2  billion 
and  90,000  man-years  of  student  time  were  devoted  to  initial  voca- 
tional training. 

Because  of  its  large  cost  a great  deal  of  attention  has  been 
given  to  Improving  the  efficiency  of  initial  specialty  training.  Re- 
search in  this  area  has  focused  almost  exclusively  on  one  aspect  of 
training  efficiency — technical  efficiency.  That  is,  researchers  have 
tried  to  determine  the  mix  of  training  inputs  (curriculum  choice, 
teaching  aids,  staff,  etc.)  that  most  efficiently  will  achieve  a cer- 
tain level  of  proficiency  in  graduates. 

The  problem  of  selecting  courses  (and  corresponding  levels  of 
graduate  proficiency)  involves  another  aspect  of  training  efficiency — 
economic  efficiency.  To  evaluate  specialty  training  in  terms  of 
economic  efficiency,  one  must  compare  the  benefits  of  additional  for- 
mal training  with  the  costs  of  obtaining  those  benefits.  This  report 
describes  early  results  of  research  that  is  designed  to  address  the 
question  of  economic  efficiency  in  initial  specialty  training. 

To  evaluate  the  economic  efficiency  of  training,  one  must  assess 
the  on-the-job  performance  of  graduates  of  alternative  training  pro- 
grams. Because  additional  formal  training  is  always  more  costly,  it 
is  only  worthwhile  if  it  results  in  sufficient  improvements  in  on-the- 
job  performance  to  offset  the  additional  cost.  The  key  to  evaluating 
economic  efficiency  in  training  is  therefore  the  development  of  re- 
liable measures  of  on-the-job  performance.  These  measures  need  to 
reflect  net  rather  than  gross  productivity,  to  reflect  productivity 

* 

Estimates  of  training  costs  derived  from  Military  Manpower  Train- 
ing Report  for  FY  19763  Office  of  the  Assistant  Secretary  of  Defense 
(MR&A) , March  1975. 
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over  time  rather  than  at  a single  point,  and  to  be  based  on  the  per- 
formance of  specific  individuals  rather  than  on  the  performance  of 
a representative  individual. 

Measures  that  have  these  properties  can  be  collected  by  a variety 
of  methods,  including  direct  measurement,  job-knowledge  tests,  and 
supervisory  ratings.  Each  of  these  methods  has  advantages  and  dis- 
advantages, and  no  one  approach  is  preferable,  prima  facie,  to  the 
others. 

This  report  provides  a preliminary  analysis  of  a data  set  that 
has  been  assembled  to  explore  the  tradeoff  among  training  courses  of 
different  lengths.  The  data  consist  of  enlisted  supervisors'  estimates 
of  the  net  productivity  of  trainees  at  different  points  in  their  first 
term  of  military  service.  These  estimates  are  used  to  construct  pro- 
files of  the  time  path  of  productivity.  A number  of  productivity 
paths  are  presented  to  show  the  general  character  of  the  data  we  have 
collected. 

While  our  analysis  is  admittedly  simplistic,  the  shapes  of  and 
relationships  among  the  productivity  profiles  are  quite  consistent 
with  our  prior  expectations.  Although  further  research  is  required 
before  conclusions  can  be  drawn  regarding  efficient  mixes  between 
formal  and  on-the-job  training,  the  results  of  this  study  suggest  that 
it  is  possible  to  draw  meaningful  conclusions  from  data  based  on  the 
carefully  constructed  ratings  of  supervisors. 
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I.  INTRODUCTION1 


Almost  all  new  entrants  into  the  military  attend  a formal  course 
of  instruction  in  the  skills  required  in  their  military  occupational 
specialties.  It  is  estimated  that  about  320,000  of  the  350,000  en- 
listed personnel  joining  the  military  in  FY  1976  will  receive  such 
training.  Since  this  initial  specialty  training  will  involve  about 
80,000  man-years  of  the  trainees'  time  and  cost  about  $2  billion,2 
the  importance  of  conducting  it  efficiently  is  obvious.  Even  small 
improvements  in  efficiency  can  result  in  substantial,  recurring  savings. 

A great  deal  of  research  has  been  devoted  to  improving  "technical 
efficiency."  To  design  technically  efficient  training  courses,  it  is 
necessary  to  select  the  least-cost  mix  of  curriculum,  instructional 
staff,  teaching  aids,  etc.,  that  will  produce  graduates  having  a given 
level  of  proficiency.  There  are,  however,  an  infinite  number  of  tech- 
nically efficient  courses  in  a given  specialty,  each  one  of  which  will 
provide  its  graduates  with  a different  level  of  proficiency.  The  prob- 
lem of  selecting  among  these  courses  (and  corresponding  levels  of  pro- 
ficiency) involves  another  aspect  of  training  efficiency — economic 
efficiency.  Since  almost  any  set  of  job  skills  aould  be  taught  en- 
tirely on  the  job,  formal  specialty  training  could  be  totally  dis- 
continued without  losing  the  ability  to  maintain  an  effective  military 
force.  The  reasons  for  having  formal  specialty  training  are  economic 
rather  than  technological.  To  evaluate  the  optimal  amount  of  formal 
training,  one  must  compare  the  benefits  of  additional  training  with 
its  costs.  The  costs  of  formal  training  (faculty  and  student  salaries, 
.supplies,  etc.)  are  obvious.  The  returns  are  less  obvious  because  they 
take  the  form  of  improvements  in  trainees’  on-the-job  performance. 
Economic  efficiency  in  formal  training  is  attained  when  the  last  dollar 


This  report  was  presented  at  the  Rand  Conference  on  Defense  Man- 
power, February  4-6,  1976. 

2 

These  estimates  are  derived  from  Military  Manpower  Training  Re- 
port for  FY  197 63  Office  of  the  Assistant  Secretary  of  Defense  (Man- 
power and  Reserve  Affairs),  March  1975, 
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spent  on  increasing  the  amount  of  formal  training  increases  the  bene- 
fits received  after  training  by  one  dollar. 

Rand  is  currently  conducting  research  to  evaluate  the  cost- 
effectiveness  of  different  amounts  of  formal  initial  specialty  train- 
ing. Since  more  initial  training  is  always  more  costly  than  less, 
longer  courses  are  only  cost-effective  if  they  produce  better -qualified 
graduates.  The  key  question,  therefore,  is  how  can  the  quality  of 
graduates  be  judged?  One  way  is  to  compare  the  first-term  productivity 
of  graduates  of  different  courses.  At  a minimum,  longer  courses  must 
make  graduates  sufficiently  more  productive  during  their  first  term  to 
compensate  for  the  time  spent  in  school.  If,  for  example,  graduates 
of  a course  that  lasted  an  additional  2 months  were  no  more  productive 
than  graduates  of  a shorter  course,  the  shorter^  course  would  -clearly 
be  preferable  because  its  graduates  would  be  in  the  military  labor 
force  2 months  longer  during  their  enlistment. 

The  value  of  such  productivity  comparisons  is,  however,  limited. 
Suppose,  to  continue  the  previous  example,  that  graudates  of  the  longer 
course  proved  to  be  5 percent  more  productive  after  completing  the 
course  than  graduates  of  the  shorter  one,  even  after  allowing  for  the 
fact  that  they  had  2 months  less  on  the  job.  How  is  one  to  judge 
whether  the  increase  in  productivity  is  worth  the  extra  cost  of  the 
longer  course? 

To  analyze  such  tradeoffs,  one  must  be  able  to  measure  both  the 
costs  and  the  benefits  of  longer  courses  in  the  same  terms,  and  the 
common  denominator  with  the  most  general  applicability  is  dollars. 

If  the  analysis  is  conducted  in  these  terms,  one  can  not  only  compare 
the  efficiency  of  courses  of  different  lengths,  but  can  also  consider 
other  margins  of  substitution.  For  example,  total  first-term  train- 
ing costs  can  be  reduced  by  reducing  the  number  of  new  entrants  each 
year — perhaps  by  substituting  career  personnel  or  capital  goods  for 
first-termers — as  well  as  by  choosing  more  efficient  course  lengths. 

We  have  adopted  an  approach  to  this  problem  that  is  an  applica- 
tion of  contemporary  human  capital  theory.  Basic  to  this  apprach  is 
the  notion  that  the  costs  of  and  returns  to  training  can  be  measured 
by  comparing  an  individual's  pay  and  allowances  with  his  net  produc- 
tivity. In  the  representative  case,  the  first  term  of  service  can 
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be  thought  of  as  having  three  distinct  phases:  first,  a period  of 

basic  military  training  and  initial  technical  training;  second,  a 

period  of  on-the-job  training  when  the  value  of  the  individual’s 

3 

productivity  is  less  than  the  pay  and  allowances  he  receives;  and 
third,  a period  during  which  the  military  earns  returns  on  its  in- 
vestments in  training  because  the  person’s  productivity  is  greater 
than  his  pay  and  allowances.  These  three  periods  are  illustrated  in 
Fig.  1,  where  the  time  t^  represents  the  end  of  formal  training,  the 
time  the  end  of  the  period  of  investment  in  on-the-job  training, 
and  t^  the  end  of  the  first  term  of  service. 

During  formal  training,  the  individual’s  direct  contribution  to 
military  capability  is  by  definition  zero.  (Since  he  is  not  assigned 
to  an  operating  unit,  he  could  not  possibly  be  contributing  to  current 
military  effectiveness.)  His  net  contribution,  however,  is  negative, 
because  training  requires  resources  (instructional  staff,  classroom 
space,  etc.)  that  could  otherwise  be  contributing  to  current  military 
capability.  Therefore,  the  full  cost  of  a person’s  fo.rmal  training, 
say  at  time  t,  includes  both  the  pay  and  allowances'  he  receives  (given 
by  the  distance  AB)  and  the  opportunity  cost  of  the  other  resources 
devoted  to  his  training  (BC) . After  he  completes  formal  training,  his 
net  contribution  to  military  capability  can  be  measured  as  the  differ- 
ence between  his  direct  contribution  and  the  opportunity  cost  of  re- 
sources (such  as  the  supervisors'  time)  devoted  to  his  on-the-job 
training.  The  cost  of  on-the-job  training  at  a point  such  as  t*  can 
be  measured  by  the  difference  between  the  individual's  pay  and  allow- 
ances and  his  net  productivity  at  that  time  (in  this  case  the  distance 

4 

DE) . The  total  cost  of  on-the-job  training  (OJT)  is  approximately 


More  precisely,  this  period  is  one  of  military  investment  in  on- 
the-job  training,  since  on-the-job  training  in  the  sense  of  improve- 
ment in  job-relevant  skills  continues  as  long  as  one's  productivity  is 
increasing . 

4 

In  Fig.  1,  the  trainee's  net  productivity  is  shown  as  being  nega- 
tive immediately  after  he  completes  his  formal  training.  Negative 
productivity  would  occur  if  the  forgone  productivity  of  the  tainee’s  ' 
supervisors  exceeds  the  trainee's  direct  productivity.  Although  this 
is  not  necessarily  the  case,  it  does  appear  to  be  common. 


Time 
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the  shaded  area  in  Fig.  1.^  Finally,  there  are  returns  to  training 
whenever  the  individual's  net  productivity  exceeds  his  pay  and  allow- 
ances. These  can  be  measured  in  the  same  fashion  as  the  costs  of  on- 
the-job  training  and  are  represented  by  the  cross-hatched  area  in 
Fig.  1. 

The  difference  between  the  present  value  of  the  total  cost  and 
the  returns  to  training  constitutes  the  military's  net  investment  in 
an  individual's  first-term  training.  Naturally,  the  value  of  the  net 
investment  will  change  with  changes  in  the  length  of  formal  training. ^ 
For  example,  longer  courses  will  increase  the  cost  of  formal  training. 
The  returns  from  these  investments  are  in  the  form  of  decreases  in 
the  cost  of  on-the-job  training  and  increases  in  the  returns  to  train- 

: ing  resulting  from  enhanced  on-the-job  performance.  The  relative 

| magnitudes  of  the  two  effects  determine  whether  the  change  in  the 

i amount  of  training  is  cost-effective, 

i 

I In  general,  we  expect  the  relationship  between  length  of  formal 

I training  and  net  investment  in  training  to  be  like  that  shown  in  Fig.  2 

I 

: This  relationship  is  consistent  with  the  view  that,  initially,  ad- 
ditional amounts  of  formal  training  will  reduce  total  training  costs, 
but  that  at  some  point,  the  gains,  in  terms  of  enhanced  on-the-job 
performance,  will  not  outweigh  the  costs  of  the  additional  formal  train 
ing  and,  hence,  net  costs  will  not  be  reduced.  In  fact,  at  some  point, 
it  is  reasonable  to  assume  that  additional  amounts  of  formal  training 
will  cause  net  costs  to  rise.  The  potential  number  of  such  tradeoffs 
is,  of  course,  quite  large,  and  the  current  research  program  could  not 
attempt  to  explore  all  the  possible  alternatives.  Our  approach  to  the 
problem  is  to  use  existing  natural  experiments  in  training  lengths  in 


The  shaded  area  in  Fig.  1 represents  the  undiscounted  sum  of  on- 
the-job  training  costs.  Because  these  costs  are  incurred  at  different 
points,  they  should  be  measured  as  a discounted  sum.  Similarly,  for- 
mal training  costs  and  the  returns  to  training  should  be  measured  as 
present  values,  although,  as  a practical  matter,  formal  training 
periods  are  typically  so  short  that  discounted  and  undiscounted  values 
are  virtually  identical. 

^Assuming  that  we  are  choosing  from  among  the  set  of  technically 
efficient  courses. 
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Fig.  2 — Relationship  between  length  of  formal  training  and  net 
investment  in  training 


a variety  of  specialties  to  gauge  the  nature  of  (a)  the  potential 
economies  to  be  derived  from  more  efficient  course  length  and  (b)  the 
factors  that  influence  them. 

The  major  limitation  to  putting  this  sort  of  model  into  operation 
is  the  difficulty  of  estimating  the  time  path  of  productivity.  We  have 
developed  a method  in  which  survey  data  are  used  to  estimate  produc- 
tivity, and  we  have  recently  finished  collecting  data  on  members  of 
about  50  specialties  in  the  three  major  service  branches.  This  data 
set  consists  of  supervisory  ratings  of  the  net  contribution  to  unit 
production  of  selected  first-term  personnel  serving  at  their  first-duty 
station.  These  ratings  apply  to  each  individual  in  the  sample  at  a 
number  of  points  during  his  first  term  of  service.  They  thus  permit 
individual  productivity  profiles  to  be  constructed  for  an  entire  first 
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term.  The  technique  used  to  elicit  these  ratings  was  that  of  a se- 
quential mail  survey.  An  initial  survey  questionnaire  was  mailed  to 
selected  first-term  personnel  identified  by  personnel  records  as  en- 
listees serving  at  their  first-duty  station.  This  survey  was  de- 
signed to  verify  sample  criteria  and  to  identify  enlisted  supervisors 
most  familiar  with  the  sampled  first-termer's  work.  A subsequent  sur- 
vey questionnaire  was  mailed  to  identified  enlisted  supervisors  to 
elicit  the  productivity  ratings  previously  described.  In  its  final 
form,  the  productivity  data  set  consists  of  background  material  on 
individual  first— term  personnel  drawn  from  service  personnel  records 
and  survey  material,  a set  of  productivity  ratings  from  an  enlisted 
supervisor  or  supervisors,  and  additional  background  material  on  the 
supervisory  raters  themselves.  This  data  set,  -then,  forms  the  basis 
for  the  productivity  analysis  of  the  general  model. 

The  purpose  of  this  report  is  to  describe  this  data  set  and  some 
of  the  results  of  our  early  analysis  of  it.  In  Section  II,  the  kinds 
of  productivity  measurements  we  have  used  are  described  in  terms  of 
the  reasons  for  choosing  them  rather  than  as  alternatives.  In  Section 
III,  we  present  estimated  productivity  curves  for  six  military  occupa- 
tional specialties  and  compare  them  to  indicate  the  types  of  curves, 
and  the  relationships  among  them,  that  resulted  from  our  data.  Sec- 
tion IV  summarizes  the  findings  of  the  study  and  indicates  the  need 
for  and  directions  of  further  research  in  this  area. 
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II.  PRODUCTIVITY  MEASUREMENT 


The  two  most  difficult  problems  associated  with  adequately  address- 
ing the  issue  of  economic  efficiency  in  military  specialty  training 
are  (1)  to  establish  an  appropriate  analytic  framework  for  comparing 
the  benefits  and  costs  of  additional  amounts  of  training  and,  since 
the  benefits  of  additional  training  take  the  form  of  increased  produc- 
tivity, (2)  to  estimate  productivity.  We  believe  that  the  human  capital 
model  just  outlined  provides  an  appropriate  framework  for  analyzing  the 
effects  of  different  amounts  of  formal  training.  In  this  section,  we 
deal  with  the  problem  of  productivity  estimation.  The  objective  is 
to  assess  the  advantages  and  disadvantages  of  several  alternative 
approaches  to  the  problem,  focusing  on  both  their  conceptual  appropri- 
ateness and  the  cost  of  gathering  the  type  of  productivity  data  needed 
to  address  the  issue  of  economic  efficiency  in  specialty  training.  The 
first  step  in  this  process  is  to  determine  the  properties  that  are  im- 
portant in  such  measures. 

PROPERTIES  OF  PRODUCTIVITY  MEASURES 

Three  properties  are  crucial  in  any  productivity  measures  used 
to  evaluate  alternative  specialty  training  policies.  The  measures 
should  (1)  permit  productivity  to  be  estimated  over  time,  not  just  at 
a single  point,  (2)  measure  net  rather  than  gross  productivity,  and 
(3)  be  linked  with  the  characteristics  of  the  individual  to  whom  they 
apply.  First,  it  is  important  that  the  time  path  of  productivity  be 
measured,  because  different  types  of  formal  training  may  affect  pro- 
ductivity differently  during  the  course  of  a military  career.  For 
example,  if  one  were  comparing  two  equally  costly  training  courses  in 
which  one  course  emphasized  the  skills  needed  by  a mature  technician 
and  the  other  emphasized  the  skills  that  an  enlistee  would  use  in  the 
early  months  of  his  first-duty  assignment,  a measure  that  captured 
only  productivity  in  the  early  months  after  completion  of  training 
would  favor  the  latter  course,  whereas  a measure  that  focused  on  pro- 
ductivity later  in  the  career  would  favor  the  former.  Adequate  evalu- 
ation requires  comparisons  of  productivity  at  a number  of  points  so 
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that  comparisons  can  be  made  among  courses  with  differential  effects 
on  productivity  over  time. 

Second,  it  is  important  that  net  rather  than  gross  productivity 
be  measured.  An  individual's  gross  productivity  is  the  amount  he 
personally  produces;  his  net  productivity  is  the  difference  between 
the  unit's  production  in  his  presence  and  in  his  absence.  The  two 
need  not  be  at  all  the  same,  and  the  relationship  between  them  can  be 
expected  to  change  systematically  with  experience.  Consider,  for 
example,  a new,  specialty  school  graduate  who  joins  a large  radj.o 
repair  shop.  Although  he  will  probably  be  able  to  complete  some  sim- 
ple types  of  repairs,  he  will  almost  certainly  need  fairly  close 
supervision.  His  gross  productivity  in  these  circumstances  is  posi- 
tive, but  if  the  reduction  in  his  supervisors'  production  exceeds  the 
trainee's  direct  production,  his  net  productivity  is  negative.  As  he 
acquires  more  experience,  his  gross  productivity  will  normally  in- 
crease, and  the  amount  of  supervision  he  requires  will  decrease,  so 
that  net  productivity  will  rise  more  rapidly  than  gross  productivity. 

At  some  point,  supervision  will  become  minimal,  and  gross  and  net  pro- 
ductivity will,  for  all  intents  and  purposes,  be  identical. ^ To  the 
extent  that  the  two  measures  differ,  net  productivity  is  clearly  the 
appropriate  concept  for  evaluating  training  because  it  measures  the 
difference  in  military  effectiveness  attributable  to  an  individual. 

The  use  of  gross  productivity  tends  to  bias  analyses  of  substitutions 
between  formal  and  on-the-job  training  in  favor  of  on-the-job  training 
by  overstating  the  trainee's  productivity  during  his  early  experience 
when  supervisory  inputs  are  greatest. 

The  ability  to  relate  productivity  measures  and  trainee  character- 
istics, such  as  aptitude,  education,  attitudes,  etc.,  is  essential  to 
a detailed  analysis  of  training  alternatives.  Otherwise,  it  is  ex- 
tremely difficult  to  control  for  personal  characteristics  in  assessing 
the  effects  of  alternative  amounts  of  formal  training.  Lacking  these 
relationships,  one  is  essentially  restricted  to  making  comparisons 

_ 

Of  course,  for  those  with  substantial  supervisory  responsibilities, 
the  relationship  is  reversed  and  net  productivity  exceeds  gross  produc- 
tivity. This  circumstance  is  not  likely  to  be  relevant  during  the 
period  of  interest  here. 


-10- 


araong  representative  Individuals  or  to  performing  experiments  involv- 
ing matched  samples.  The  former  approach  is  undesirable  because  one 
can  never  be  certain  how  representative  the  "representative  indi- 
vidual" is.  The  latter  approach  is  undesirable  because  it  precludes 
the  use  of  natural  experiments  and  because  the  experiment  in  this 
case  (including  formal  and  on-the-job  training)  is  quite  lengthy. 

Both  approaches  are  limited  in  the  sense  that  the  relationships  that 
are  estimated  are  conditional  on  a given  mix  of  trainee  attributes — 
they  provide  no  information  on  the  effects  of  changes  in  trainee 
attributes. 

ALTERNATIVE  MEASUREMENT  PROCEDURES 

Measures  having  the  properties  just  discus'sed  can  be  gathered  in 
a number  of  ways.  Here  we  describe  the  strengths  and  weaknesses  of 
two  of  the  major  alternatives,  which  provides  a basis  for  evaluating 
the  measurement  procedure  we  have  used. 

Substitution  Measurement 

The  general  character  of  the  productivity  measures  one  would  col- 
lect with  unlimited  resources  is  fairly  straightforward.  Such  measures 
would  involve  estimating  an  individual’s  net  productivity  by  measuring 
a unit’s  output  in  his  presence  and  in  his  absence.  In  our  example  of 
the  radio  repair  shop,  the  shop's  output  could  be  measured  with  its 
full  complement  of  personnel  and  then  with  various  combinations  of 
n - 1 personnel.  The  difference  between  the  output  of  repairs  with 
and  without  a given  person  is  a measure  of  his  net  productivity.  A 
time  path  of  net  productivity  could  be  estimated  either  cross-section- 
ally  or  longitudinally. 

One  substantial  difficulty  in  implementing  this  approach  is  the 
measurement  of  unit  output.  The  problem  arises  primarily  because  there 
are  many  different  types  of  output  in  a given  specialty.  To  return  to 
the  radio  shop  example:  Within  a given  shop,  several  types  of  radios 
will  be  maintained  and  many  types  of  failures  will  occur.  If  there  are 
substantial  variations  in  the  difficulty  of  repairing  different  types 
of  failures  and  in  the  mix  of  failures  over  time,  the  number  of  "re- 
paired radios"  that  can  be  turned  out  in  a given  number  of  man-hours 
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will  differ  considerably.  To  take  these  variations  into  account, 
weights  must  be  developed  for  different  types  of  repairs,  and  output 
must  be  measured  as  a weighted  sum  of  repairs.  When  the  context  is 
broadened  to  include  multiple  shops,  the  development  of  appropriate 
weights  is  even  more  important,  because  differences  in  equipment,  mixes 
among  shops  can  introduce  substantial  differences  in  measured  produc- 
tivity if  inappropriate  weights  are  selected. 

At  first  glance  it  may  seem  that  once  an  unambiguous  definition 
of  output  is  developed,  a small  number  of  observations  will  be  suf- 
ficient to  evaluate  two  alternative  training  strategies  because  the 
productivity  measurements  can  presumably  be  made  quite  precisely. 

This  is  not  true,  however,  because  of  the  large  number  of  factors 
besides  military  training  that  influence  a person’s  contribution  to 
unit  performance.  These  include  (a)  his  motivation,  ability,  previous 
education,  etc.;  (b)  the  number,  experience  level,  motivation,  ability, 
and  previous  education  of  other  personnel  in  the  unit;  (c)  the  group's 
experience  in  working  together;  (d)  the  stock  of  capital  equipment; 
and  (e)  the  demand  for  service  that  the  shop  faces.  To  control  for 
these  factors,  a large  number, of  observations  are  necessary. 

It  should  be  apparent  from  the  preceding  discussion  that  the 
"ideal"  sort  of  productivity  measures  would  be  quite  costly  to  assemble 
and  would,  in  spite  of  the  cost,  be  less  than  perfect.  This  suggests 
that  it  is  worthwhile  to  consider  other  alternatives.  If  suitable 
less  costly  alternatives  can  be  found,  they  would  be  preferable  if 
only  because  of  the  limited  amount  of  previous  research  in  this  area. 
Since  the  information  available  is  so  limited,  it  is  probably  better 
to  obtain  a first  approximation  of  productivity  and  economic  efficiency 
of  training  for  a number  of  specialties  than  to  devote  the  same  amount 
of  resources  to  a detailed  analysis  of  one  or  two  specialities. 

Job-Performance  Tests 

One  approach  that  retains  the  characteristic  of  direct  measure- 
ment but  involves  measures  that  are  simpler  to  develop  and  administer 
is  the  use  of  job-performance  tests.  This  approach  involves  testing 
individuals  on  a specific  set  of  skills  used  in  their  specialty.  By 
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testing  people  with  different  amounts  of  on-the-job  experience,  or  one 
person  at  several  times,  the  relationship  between  productivity  and 
experience  can  be  established;  and  by  linking  these  measurements  to 
those  of  formal  technical  training,  background,  ability,  etc.,  one 
can  estimate  the  effects  of  different  initial  training  (controlling 
for  personal  characteristics). 

There  are  several  major  limitations  to  this  approach,  however. 

Most  importantly,  it  involves  measurement  of  gross,  rather  than  net, 
productivity.  The  on-the-job  training  process  in  the  military  gen- 
erally involves  substantial  inputs  of  supervisory  resources,  and  there- 
fore differences  between  net  and  gross  productivity  are  likely  to  be 
large,  especially  early  in  the  on-the-job  training  process.  Use  of 
gross  productivity  will  result  in  downward-biased  estimates  of  on-the- 
job  training  costs  and,  therefore  (because  the  returns  to  formal  train- 
ing are  understated) , will  lead  to  policies  that  entail  less-than- 
efficient  amounts  of  formal  training.  A more  subtle  bias  occurs  be- 
cause of  differentials  in  supervision  across  individuals.  One  would  ex- 
pect the  level  of  supervisory  inputs  to  the  on-the-job  training  process 
to  vary  with  the  trainee’s  amount  of  formal  training  and  his  personal 
attributes.  If,  for  example,  better  trained,  more  able  personnel  re- 
quire less  supervision,  gross  productivity  measures  understate  the 
differential  in  performance  between  the  better-trained,  more  able 
individuals  and  those  with  less  ability  and  training.  This  implies 
that  gross  productivity  measures  will  fail  to  capture  part  of  the  re- 
turns to  additional  formal  training  and  also  that  the  relationships 
between  personal  characteristics  and  productivity  estimated  from  gross 
productivity  data  will  be  biased.  Finally,  there  are  real  questions 
of  how  well  job-performance  tests  measure  actual  gross  productivity. 

Even  if  the  chosen  job  tasks  accurately  reflect  the  duties  in  a par- 
ticular specialty,  they  may  not  accurately  represent  a particular  in- 
dividual’s actual  duties.  Further,  job  tests  measure  a person’s  capa- 
bility to  perform  those  tasks,  not  his  actual  performance  of  them. 

The  difference  in  the  observed  performance  of  two  similar  persons  with, 
for  example,  different  levels  of  motivation  is  likely  to  be  much  smaller 
during  a short  test  than  over  the  course  of  several  days  or  weeks  on 
the  job. 
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Rand  Method 

The  approach  we  have  chosen  uses  supervisors'  ratings  of  net 
productivity  rather  than  direct  measurement.  This  approach  has  the 
disadvantage  of  being  a subjective  measure,  but  it  also  has  many  ad- 
vantages, especially  since  there  has  been  so  little  previous  analysis 
of  the  relationship  between  formal  specialty  training  and  productivity. 
In  the  following  discussion,  we  will  describe  the  productivity  measure- 
ments that  have  been  gathered  and  examine  the  strong  and  weak  points 
of  the  approach. 

We  have  used  a self-administered  mail  questionnaire  to  obtain 
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supervisors'  ratings  of  the  net  productivity  of  specific  trainees  at 
several  points  during  their  first  term  of  service.  Specifically, 
supervisors  were  asked  to  rate  each  trainee's  productivity  (1)  during 
his  first  month  with  the  unit,  (2)  at  the  time  that  the  rating  was 
completed,  (3)  1 year  from  the  time  of  rating,  and  (4)  after  completion 
of  4 years  of  service.  In  each  instance,  the  supervisor  was  asked  to 
rate  the  individual's  net  productivity  ralative  to  that  of  the  typical 
specialist  with  4 years  of  experience.  Together,  these  points  trace 
out  an  estimated  time  path  of  relative  net  productivity  that  can  be 
related  to  the  attributes  of  the  person  being  rated. 

This  method  of  estimating  productivity  is  attractive  for  four 
reasons.  First,  it  measures  net  rather  than  gross  productivity. 

Second,  it  does  permit  one  to  control  for  personal  characteristics  in 
comparing  training  alternatives.  Third,  the  cost  of  data  collection 
is  relatively  low,  which  means  that  for  a given  budget  many  more 
specialties  can  be  analyzed  than  would  be  possible  using,  for  example, 
substitution  measures.  Although  substitution  measures  would  presumably 
lead  to  more  precise  estimates,  it  appears  more  valuable,  at  the  present 
stage  of  research  In  this  area,  to  explore  the  general  magnitude  amd 
pattern  of  training  effects  across  a number  of  specialties  than  to 
analyze  a small  number  of  specialties  in  great  detail.  Fourth,  be- 
cause of  the  general  nature  of  the  measures  that  are  obtained,  compar- 
isons across  specialties  are  feasible  both  within  a given  service  and 


Net  productivity  is  defined  as  the  difference  between  a trainee's 
gross  productivity  and  the  forgone  productivity  of  supervisors  who 
work  with  him. 
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across  services.  This  feature  is  also  important  in  obtaining  a broad 
overview  of  the  effects  of  training  on  productivity. 

Two  important  limitations  to  our  approach  should  be  considered. 
First,  the  concept  of  net  productivity  is  fairly  sophisticated,  and 
it  is  not  one  that  enlisted  supervisors  are  likely  to  have  been  famil- 
iar with  before  receiving  the  questionnaire.  Thus  it  is  possible  that 
some  survey  responses  will  be  invalid  because  supervisors  did  not  under- 
stand what  they  were  being  asked  to  do.  Because  of  this  possibility, 
we  spent  a great  deal  of  time  and  effort  in  the  survey  design  stage 
to  develop  and  field  test  a clear  explanation  of  the  concept  of  net 
productivity.  Of  course,  no  such  explanation  could  be  clear  to  all 
recipients  of  the  questionnaire,  so  a simple  test  of  comprehension  was 
included  in  the  survey  instrument.  In  addition,  to  their  questionnaire 
responses,  supervisors  were  asked  to  rate  both  a "typical"  technical 
school  graduate  and  a directed  duty  assignment  trainee;  this  rating 
was  intended  to  provide  some  insight  into  the  supervisor's  comprehen- 
sion of  the  concept.  Clearly,  some  responses  will  still  be  unusable 
because  of  the  rater's  inability  to  comprehend  the  concept  of  net  pro- 
ductivity, but  a preliminary  analysis  of  the  data  does  not  indicate 
that  this  is  a serious  problem. 

Another  potential  limitation  is  the  possibility  of  important  dif- 
ferences among  raters  in  their  rating  systems — some  tend  to  rate  easy 
and  others  hard;  some  tend  to  see  people  as  very  similar  and  some  as 
very  dissimilar.  These  differences  have  the  potential  of  producing 
substantial  noise  in  the  data,  and  a pilot  study  of  our  approach^ 
suggested  that  they  were  indeed  a serious  consideration.  An  econo- 
metric model  for  dealing  with  this  source  of  distortion  by  estimating 
parameters  of  the  supervisor's  rating  system  was  developed  in  conjunc- 
tion with  the  pilot  study,  and  has  been  expanded  and  tested  through 
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A directed  duty  assignment  trainee  is  one  who  goes  directly  from 
basic  military  training  to  an  operating  unit  without  attending  a for- 
mal specialty  training  school. 

10R.  M.  Gay,  Estimating  the  Cost  of  On-tke-Job  Training  in  Military 
Occupations:  A Methodology  and  Pilot  Study s The  Rand  Corporation, 
R-1351-ARPA,  April  1974. 
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Monte  Carlo  simulation.  It  provides  a means  for  controlling  for 
supervisor-rating  effects.  The  ratings  of  a typical  trainee  provide 
another  method  of  determining  distortion,  since  one  would  expect,  for 
example,  the  supervisor  who  is  a harder-than-average  rater  to  give  both 
the  "typical  trainee"  and  individual  trainees  low  ratings.  A pre- 
liminary analysis  of  our  data  suggests  that  a good  deal  of  the  noise 
in  the  supervisor  ratings  is,  in  fact,  eliminated  by  controlling  for 
the  supervisor's  rating  of  the  typical  trainee. 


Richard  V.L.  Cooper  and  Gary  R.  Nelson,  Analytic  Methods  for 
Adjusting  Subjective  Rating  Schemes  3 The  Rand  Corporation,  R-1685- 
ARPA,  June  1976. 
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III.  PRODUCTIVITY  RELATIONSHIPS 


The  data  base  assembled  at  Rand  to  provide  a vehicle  for  analyzing 
the  effects  of  different  amounts  of  first-term  specialty  training  is 
very  large  and,  in  many  respects,  unique.  It  includes  information1 on 
Individuals  in  more  than  50  specialties  in  the  three  major  service 
branches.  When  preliminary  data  processing  is  completed,  an  observa- 
tion will  include  (a)  the  supervisor’s  rating  of  a trainee,  (b)  back- 
ground information  from  service  personnel  records  on  both  the  trainee 
and  the  supervisor,  and  (c)  additional  background  information  on  the 
trainee  obtained  from  a survey  he  had  completed.  The  survey  data  were 
collected  through  sequential  mail  surveys  of  about  30,000  first-term 
enlisted  personnel  and  an  approximately  equal  number  of  enlisted  super- 
visors. This  section  presents  the  findings  of  a very  preliminary 
analysis  of  these  data.  Because  we  are  at  a very  early  stage  in  our 
work,  we  have  adopted  a rather  elementary  form  of  analysis---compari- 
sons  of  estimated  productivity  functions  over  time  in  several  special- 
ties. In  presenting  these  results,  we  intend  to  suggest  a broad  con- 
sistency between  the  observed  patterns  of  productivity  and  those  one 
would  have  expected  a priori. 

This  section  is  divided  into  three  parts.  First,  we  describe  the 
data  used  in  the  present  analysis.  Second,  we  present  a comparison 
of  estimated  productivity  curves  for  six  military  occupational  special- 
ties. Third,  by  analyzing  estimated  productivity  curves  for  the  "typ- 
ical" technical  school  graduate  and  directed  duty  assignment  trainee, 
we  illustrate  both  the  potential  for  and  the  pitfalls  of  using  these 
sorts  of  data  to  analyze  effects  of  training  on  productivity. 

DATA  CHARACTERISTICS 

As  mentioned  previously,  our  productivity  data  consist  of  re- 
sponses to  questionnaires  administered  by  mail  to  supervisors  in  a 
selected  set  of  military  occupational  specialties.  Supervisors  were 
requested  to  provide  three  types  of  estimates  of  net  productivity  over 
time.  First,  and  most  important,  we  asked  for  estimates  of  specific 
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individuals ' net  contribution  to  unit  productivity  at  four  points 
during  their  first  term  of  service:  (1)  during  the  first  month  on 
the  job,  (2)  at  the  time  that  the  rating  was  completed  (which  will, 
of  course,  imply  different  amounts  of  on-the-job  experience  for  dif- 
ferent individuals),  (3)  1 year  from  the  time  of  the  rating,  and  (4) 
after  4 years  of  service.  Second,  we  asked  for  estimates  of  the  typ- 
ical technical  school  graduate’s  net  contribution  to  unit  production 
during  his  first  month  on  the  job,  after  1 year  on  the  job,  after  2 
years  on  the  job,  and  after  4 years  of  service.  Third,  we  asked  for 
estimates  of  the  typical  directed  duty  assignment  trainee's  net  con- 
tribution to  unit  production  after  1 month,  1 year,  and  2 years  on 
the  job  and  after  4 years  of  service. 

For  purposes  of  the  present  analysis,  we  have  selected  a sub- 
sample of  6 of  the  more  than  50  military  occupational  specialties  for 
which  data  have  been  collected.  The  subsample  covers  a range  of  job 
tasks  and  includes  a set  of  comparable  specialties  in  different  serv- 
ices. We  have  selected  the  light  weapons  infantry  specialists  (11B) 
and  the  hospital  corpsmen  (91B)  in  the  Army;  radio  repair  specialists 
(ETN)  and  hospital  corpsmen  (HM)  in  the  Navy;  radio  repair  specialists 
(304X4)  and  hospital  corpsmen  (902X0)  in  the  Air  Force.  Observations 
for  an  individual  were  included  only  if  a complete  set  of  four  produc- 
tivity estimates  was  available,  and  if  the  trainee  (a)  was  serving  at 
his  first-duty  station,  (b)  had  attended  technical  school  training, 
and  (3)  was  serving  in  the  specialty  in  which  he  was  trained.  Further, 
to  eliminate  responses  clearly  indicating  that  the  supervisor's  com- 
prehension of  the  concepts  was  poor,  we  deleted  cases  in  which  the 
supervisor  rated  the  typical  technical  school  graduate's  productivity 

at  100  percent  or  more  during  his  first  month  on  the  job  or  at  zero 

12 

or  less  after  4 years  of  service. 

COMPARISON  OF  PRODUCTIVITY  CURVES  FOR  SIX  SPECIALTIES 

Representative  productivity  curves  for  members  of  a given  specialty 
can  be  estimated  by  using  ratings  of  the  productivity  of  specific 

12 

Using  these  criteria,  the  number  of  cases  in  each  specialty  were 
11B,  96;  91B,  197;  ETN,  252;  HM,  85;  304X4,  689;  and  902X0,  363. 
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individuals,  as  illustrated  for  two  Navy  occupational  specialties 
(radio  repairmen  and  hospital  corpsmen)  in  Fig.  3.  Average  produc- 
tivity during  the  first  month  on  the  job  and  after  4 years  of  service 
(points  1 and  8 in  Fig.  3)  can  be  estimated  by  taking  the  average  value 
of  the  responses  for  all  individuals  in  the  sample.  Intermediate 
points  are  estimated  by  taking  the  average  estimate  of  productivity  at 
the  present  time  and  1 year  from  now  for  groups  of  individuals  in  dif- 
ferent experience  categories.  Specifically,  we  have  grouped  individuals 
into  three  experience  categories:  5-9,  10-14,  and  15-19  months  of  job 
experience.  Points  2-4  in  Fig.  3 represent  the  mean  values  of  the 
estimates  of  productivity  at  the  time  of  the  rating  for  those  in  each 
of  the  categories  (plotted  at  the  midpoints  of  the  intervals) . Points 
5-7  represent  the  average  values  of  estimated  productivity  a year  from 
the  time  of  the  rating  for  people  in  the  same  experience  categories 

On  average,  these  estimates  of  relative  productivity  for  both  the 
radio  repairmen  and  hospital  corpsmen  conform  to  our  expectations. 

The  curves  have  positive  slopes,  reflecting  an  increase  in  productivity 
as  a function  of  experience.  Further,  estimated  relative  productivity 
after  4 years  of  experience  approaches  100  percent  in  both  specialties. 
Since  productivity  is  being  measured  relative  to  the  average  specialist 

with  4 years  of  experience,  average  productivity  at  this  point  would, 

15 

in  the  absence  of  bias,  be  100  percent.  The  averaged  ratings  of  the 
radio  repairmen  are  consistently  below  those  for  the  hopsital  corps- 
men,  which  is  to  be  expected  because  radio  repair  is  more  technically 
demanding  and  requires  more  experience  before  one  can  attain  proficiency 
Figure  4 shows  similar  comparisons  for  the  Air  Force  hospital  corpsmen 
and  radio  repairmen,  whose  productivity  patterns  are  similar  to  those 
of  the  Navy  specialists.  Initial  net  productivity  is  negative  for  radio 
repairmen  and  positive  for  hospital  corpsmen,  and  the  differences  in 


Since  the  horizontal  axis  of  Fig.  3 measures  on-the-job  expe- 
rience and  point  8 corresponds  to  productivity  after  4 years  of  service, 
the  position  of  point  8 in  a given  specialty  depends  on  the  length  of 
formal  technical  training  in  that  specialty. 

14 

Cells  with  fewer  than  10  observations  were  not  included  in  the 

plots. 

■^This  at  least  partly  explains  why  the  difference  between  the  two 
curves  diminishes  over  time. 


Relative  productivity  (percent) 
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Fig.  3 — Comparison  of  productivity  profiles  in  two  Navy  specialties: 
hospital  corpsman  (HM)  and  radio  repairman  (ETN) 


Fig. 4 — Comparison  of  productivity  profiles  in  two  Air  Force  specialties: 
hospital  corpsman  (902X0)  and  radio  repairman  (304X4) 
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relative  productivity  diminish  with  experience.  The  curves  for  the 
two  Army  specialties  (hospital  corpsmen  and  light  weapons  infantry- 
men) shown  in  Fig.  5 are  less  clear-cut.  Here  it  is  not  so  obvious 
which  is  the  more  technically  demanding  specialty;  moreover,  the  dif- 
ferential in  productivity  does  not  decline  over  time  as  expected. 


Another  way  of  exploring  the  consistency  of  our  estimates  is  by 
comparing  estimates  of  similar  specialties  in  different  services.  The 
two  curves  in  Fig.  6 show  the  productivity  profiles  in  two  radio  re- 
pair specialties,  Navy  ETN  and  Air  Force  AFSC  304X4.  Here  a striking 
similarity  can  be  observed  in  both  initial  and  final  productivity  and 
rate  of  progress.  The  curves  suggest  that  there  is  a high  degree  of 
comparability  between  the  two  services  in  this  occupational  specialty. 

] Figure  7 makes  a similar  comparison  of  the  productivity  ratings 

i 

■for  hospital  corpsmen  in  the  three  services.  Again  the  consonance 

i 

among  the  curves  is  striking  in  terms  of  both  slopes  and  intercepts. 
However,  the  average  productivity  curve  for  the  Air  Force  corpsmen  has 
a slightly  lower  initial  value  than  the  curves  for  the  Army  and  Navy 


Relative  productivity  (percent)  Relative  productivity  (percent) 
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Fig.  6 — Comparison  of  productivity  profiles  in  two  radio  repair  specialties: 
Navy  ETN  and  Air  Force  AFSC  304X4 


Fig.  7 Comparison  of  productivity  profiles  in  three  hospital  corpsman  specialties: 
Army  91 B,  Navy  HM,  and  Air  Force  902X0 
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corpsmen.  One  possible  explanation  is  that  hospital  corpsmen  in  the 
Air  Force  receive  approximately  20  percent  less  formal  schooling  than 
those  in  the  Army  and  Navy,  but  this  hypothesis  cannot  be  tested  in 
the  current  analytic  framework. 

Servicewide  productivity  curves  for  both  hospital  corpsmen  and 
radio  repairmen  are  presented  in  Fig.  8.^  The  relationships  in  this 
instance  are  quite  similar  to  those  observed  for  the  Navy  and  Air  Force 
in  Figs.  3 and  4. 


Fig.  8 — Servicewide  comparison  of  two  occupational  specialties: 
hospital  corpsman  and  radio  repairman 


As  a group,  these  results  are  quite  encouraging.  They  are  con- 
sistent with  expectations  both  in  terms  of  the  productivity  patterns 
observed  over  time  in  a given  specialty  and  the  relationships  among 
specialties.  These  findings  suggest  that  the  productivity  estimates 
we  have  assembled  will  be  useful  in  appraising  alternative  training 
policies. 


These  curves  include  all  members  of  the  respective  specialties 
regardless  of  service. 
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COMP ARISON  OF  PRODUCTIVITY  CURVES  FOR  GRADUATES  AND  NONGRADUATES 

Both  the  potential  for  and  the  pitfalls  of  using  these  types  of 
data  to  analyze  the  effects  of  training  on  productivity  can  be  illus- 
trated by  some  comparisons  of  productivity  curves  estimated  as  aver- 
ages of  supervisors’  ratings  of  the  '’typical”  technical  school  gradu- 
ate’s performance  and  the  performance  of  the  "typical”  directed  duty 
assignment  trainee.  Unlike  the  curves  estimated  from  ratings  of  spe- 
cific individuals,  these  curves  chart  the  progress  of  hypothetical 
typical  trainees  through  the  first  term  of  service. 

The  four  curves  in  Fig.  9 present  such  comparisons  for  two  radio 
repair  specialties:  Navy  ETN  and  Air  Force  304X4.  The  curves  exhibit 
properties  consistent  with  the  general  expectations  of  productivity 
estimates  and  are  similar  to  the  average  curves*  estimated  for  specific 
individuals.  The  curves  for  the  Navy  radio  repairmen  (represented  by 
the  solid  lines)  show  substantial  and  persistent  differences  between 


Fig.  9 — Comparison  of  combined  producHvify  profiles  of  "typical"  graduate 
and  nongraduate  trainees  in  tv/o  radio  repair  specialties: 

Navy  ETN  and  Air  Force  304X4 
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the  performance  of  graduates  and  nongraduates  throughout  the  first 
4 years  of  service  and  suggest  that  returns  to  training  may  be  sub- 
stantial. The  curves  for  the  Air  Force  radio  repairmen  (represented' 
by  the  dotted  lines)  show  a similar  relationship  between  the  perfor- 
mance of  graduates  and  nongraduates,  but  the  differences  appear  to 
be  much  smaller  than  those  between  the  two  Navy  curves. 

Based  on  these  two  sets  of  curves  alone,  one  might  argue  that 
there  is  a difference  in  training  effectiveness  between  the  Navy  and 
Air  Force,  with  higher  returns  to  training  occurring  in  the  Navy. 
Consideration  of  the  two  sets  of  curves  together,  however,  raises 
questions  about  this  interpretation.  The  two  curves  for  the  graduate 
repairmen  are  practically  identical;  the  differences  between  the 
services  arise  because  Navy  nongraduates  do  not  perform  as  well  as 
Air  Force  nongraduates.  This  suggests  that  the  apparent  difference 
in  returns  to  training  between  the  two  services  may,  in  fact,  be  due 
to  differences  in  the  relative  quality  of  graduates  and  nongraduates 
rather  than  to  differences  in  training  effectiveness.  That  is,  Navy 
and  Air  Force  technical  school  graduates  may  be  roughly  comparable  in 
terms  of  aptitude,  education,  etc.,  but  Navy  directed  duty  assignment 
trainees  may  be  less  able  than  their  Air  Force  counterparts.  Of  course, 
with  this  sort  of  analysis  one  cannot  really  sort  out  the  effects  of 
differences  in  personal  characteristics  from  differences  in  training 
effectiveness.  The  results  do  imply,  however,  that  we  are  likely  to 
observe  systematic  differences  in  the  performance  of  graduates  and 
nongraduates,  and  that  in  analyzing  them  it  will  be  important  to  con- 
trol for  the  personal  characteristics  of  trainees. 


-25- 


IV . CONCLUS IONS 


To  determine  the  efficient  amount  of  technical  school  training 
for  military  occupational  specialties,  the  analysis  must  be  conducted 
within  a framework  that  permits  the  analyst  to  estimate  the  effects 
of  that  training  on  posttraining  job  performance.  The  approach  we 
have  chosen  is  an  application  of  human  capital  theory  in  which  the 
costs  of  on-the-job  training  and  the  returns  to  training  are  measured 
by  comparing  the  trainee’s  pay  and  net  productivity.  The  key  element 
in  implementing  this  approach  is  the  estimation  of  on-the-job  produc- 
tivity. To  construct  estimates  of  the  time  path  of  productivity,  we 
have  used  supervisors'  estimates  of  the  net  productivity  of  trainees 
at  various  points  in  their  first  term  of  service.  A number  of  esti- 
mated productivity  paths  were  presented  to  show  the  general  character 
of  the  data  we  have  collected. 

All  the  productivity  curves  for  the  six  specialties  discussed  in 
this  report  showed  a positive  slope  that  tended  to  decrease  over  time 
(the  rate  of  improvement  declined  with  experience).  Moreover,  com- 
parisons among  specialties  tended  to  conform  to  expectations,  both 
when  those  comparisons  were  made  between  specialties  in  the  same 
service  and  when  they  were  made  between  comparable  specialties  across 
services. 

Encouraging  as  these  results  are,  they  are  of  no  direct  help  in 
solving  the  problem  of  efficient  amounts  of  technical  school  train- 
ing. Two  steps  are  necessary  before  this  problem  can  be  addressed. 
First,  the  productivity  estimates  must  be  integrated  into  a broader 
framework  so  that  training  costs  and  returns  to  training  can  be  esti- 
mated. Second,  estimated  net  training  costs  (formal  training  costs 
plus  on-the-job  training  costs  less  returns  to  training)  must  be 
analyzed  for  individuals  with  differing  amounts  of  formal  training. 
And,  as  the  results  of  this  study  of  "typical  trainee"  data  suggest, 
future  analyses  must  be  done  in  a multivariate  framework  that  controls 
for  other  factors  that  influence  estimated  productivity. 
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